Papers with embedding methods
On the Influence of Coreference Resolution on Word Embeddings in Lexical-semantic Evaluation Tasks (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing word embeddings rely on local information delimited by context windows or dependency parents to predict word relations. |
| Approach: | They propose to use coreference resolution to find all spans of a text that refer to the same entity to improve the F1-Scores. |
| Outcome: | The proposed methods do not benefit significantly from pronoun substitution. |
Towards Understanding the Geometry of Knowledge Graph Embeddings (P18-1)
Copied to clipboard
| Challenge: | Knowledge Graph (KG) embedding has emerged as a very active area of research over the last few years, resulting in the development of several embeddable methods. |
| Approach: | They propose to use KG embedding methods to represent entities and relations as vectors in a high-dimensional space. |
| Outcome: | The proposed methods represent entities and relations in KGs as vectors in a high-dimensional space. |
Relation Embedding with Dihedral Group in Knowledge Graph (P19-1)
Copied to clipboard
| Challenge: | Existing models for link prediction fail to capture relation compositions, resulting in lack of interpretability for reasoning on incomplete knowledge graph (KG). |
| Approach: | They propose a new model that learns knowledge graph embeddings that can capture relation compositions by nature and reduces the solution space drastically. |
| Outcome: | The proposed model outperforms existing models and is comparable to or better than deep learning models such as ConvE. |
Evaluating the Effects of Embedding with Speaker Identity Information in Dialogue Summarization (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for automatic dialogue summarization do not take into account speaker identity information, but instead use sinusoidal functions to embed speaker information at the less informative part of the position embedding. |
| Approach: | They propose to embed speaker identity information into a dialogue transcript encoder to address this issue and reduce the "who said what"-related errors. |
| Outcome: | The proposed method improves the convergence of the model in training and increases the average ROUGE scores of the generated summaries in comparison to existing methods. |
Building Location Embeddings from Physical Trajectories and Textual Representations (2020.aacl-main)
Copied to clipboard
| Challenge: | Using a dataset consisting of the location trajectories of 729 students over a seven month period, we investigate whether embeddings can represent aspects such as location presence or location functionality. |
| Approach: | They propose to use location embeddings to generate embeddables of sequences of locations a student has visited to identify surface properties captured in the representations. |
| Outcome: | The proposed models can be used to predict depression levels and area of study, and can be applied to complex tasks such as predicting area of studies and depression levels. |
AutoML Meets Hugging Face: Domain-Aware Pretrained Model Selection for Text Classification (2025.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning text classification models are resource-intensive and require substantial computational power and time. |
| Approach: | They propose a corpus-driven domain mapping pipeline that integrates pre-fine-tuned models from Hugging Face Model Hub into AutoML systems to improve model selection. |
| Outcome: | The proposed pipeline improves model selection and streamlines workflows and reduces computational costs. |
Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to generalize knowledge bases model triple-level uncertainty . Existing models only model triple level uncertainty, and reasoning results lack global consistency. |
| Approach: | They propose a method to embed knowledge graphs with calibrated probabilistic semantics . they model each entity as a box and relations between two entities as affine transforms based on affinity transforms. |
| Outcome: | Experiments show that the proposed method outperforms baseline methods on confidence prediction and fact ranking. |
PoD: Positional Dependency-Based Word Embedding for Aspect Term Extraction (2020.coling-main)
Copied to clipboard
| Challenge: | Existing word embeddings that capture the contextual information only produce moderate results in aspect term extraction. |
| Approach: | They propose a positional dependency-based word embedding which takes both dependency context and positional context into account for aspect term extraction. |
| Outcome: | The proposed method outperforms other embedding methods in aspect term extraction. |
AutoRAG-HP: Automatic Online Hyper-Parameter Tuning for Retrieval-Augmented Generation (2024.findings-emnlp)
Copied to clipboard
Jia Fu, Xiaoting Qin, Fangkai Yang, Lu Wang, Jue Zhang, Qingwei Lin, Yubo Chen, Dongmei Zhang, Saravan Rajmohan, Qi Zhang
| Challenge: | Recent advances in Large Language Models have transformed ML/AI development . a reevaluation of AutoML principles for Retrieval-Augmented Generation (RAG) systems is needed. |
| Approach: | They propose a framework for hyper-parameter tuning and a hierarchical MAB method for efficient exploration of large search spaces. |
| Outcome: | The proposed framework outperforms baseline methods in more challenging optimization scenarios. |
Hyperbolic Hierarchy-Aware Knowledge Graph Embedding for Link Prediction (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding methods are built on Euclidean space, which are difficult to handle hierarchical structures. |
| Approach: | They propose a KGE model with extended Poincaré Ball and polar coordinate system to capture hierarchical structures. |
| Outcome: | The proposed model captures hierarchical relationships with extended Poincaré Ball and polar coordinate system in hyperbolic space and achieves state-of-the-art results on part of link prediction tasks. |
C-STS: Conditional Semantic Textual Similarity (2023.emnlp-main)
Copied to clipboard
Ameet Deshpande, Carlos Jimenez, Howard Chen, Vishvak Murahari, Victoria Graf, Tanmay Rajpurohit, Ashwin Kalyan, Danqi Chen, Karthik Narasimhan
| Challenge: | Semantic textual similarity (STS) is a cornerstone task in natural language processing, but it is inherently ambiguous. |
| Approach: | They propose a task called conditional STS which measures similarity conditioned on an aspect elucidated in natural language. |
| Outcome: | The proposed task reduces subjectivity and ambiguity and enables fine-grained similarity evaluation using diverse conditions. |
Traffic Light or Light Traffic? Investigating Phrasal Semantics in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Phrases are fundamental linguistic units through which humans convey semantics. |
| Approach: | They assess the capacity of API-based large language models to comprehend phrase semantics . they use three human-annotated datasets to analyze their results . |
| Outcome: | The proposed model outperforms embedding-based methods in phrase semantic reasoning tasks . the proposed model does not show significant advantage over fine-tuned methods . |
Global Voices, Local Biases: Socio-Cultural Prejudices across Languages (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on human biases are heavily skewed towards Western and European languages . despite growing interest in language models, there are several shortcomings in the literature . |
| Approach: | They scale the Word Embedding Association Test to 24 languages and add culturally relevant information for each language. |
| Outcome: | The proposed language models can reflect and often amplify the effects of bias across linguistic, cultural, and societal borders. |